# Dynamic Masked Attention

The Doge model employs dynamic masked attention mechanisms for sequence transformation, with the option to use either multi-layer perceptrons or cross-domain mixture of experts for state transitions.

The Doge model employs dynamic masked attention mechanisms for sequence transformation and can use multi-layer perceptrons or cross-domain mixture of experts for state transitions.

Doge 320M Instruct is a lightweight language model based on dynamic masked attention, trained with supervised fine-tuning (SFT) and direct preference optimization (DPO), suitable for question-answering and dialogue tasks.

Doge is a sequence transformation model that employs dynamic masked attention mechanisms, capable of state transitions using either multi-layer perceptrons or cross-domain mixture of experts.

Doge 160M Reasoning Distilled Version is a lightweight language model based on dynamic masked attention mechanism and cross-domain mixture of experts, focusing on reasoning and question-answering tasks.

Doge 160M is a small language model that employs dynamic masked attention mechanisms, trained by the SmallDoge community, and supports text generation tasks.

Doge 20M is a small language model based on dynamic masked attention mechanism, supporting instruction following and Q&A tasks.